• What is the difference/relationship between running specific states (run_state) and the transition function? Is it primarily to allow for the testing and the development? When we run the sim say over a bunch of "transitions" will we need to be sure to await the transition after running a state? What is the flow you are thinking here?

This by no means has to be the correct way to do this, but it is a working way to do things. Let's break it down for a sec and think about the Rust async model.

1. Note that run_state() is NOT async. What happens in the run_state functions is we wrap a blocking/awaited future in a tokio::task which allows tokio to take this and run it concurrently in its runtime. In essence, run_state starts running whatever state we want it to (e.g., syncing or processing). The task is placed back into the relevant struct as a JoinHandle<..> (or a JoinAll).
2. transition() is async and blocking. When we call transition(), we grab the Join, and await it to finish its execution and ultimately return the inner object back to the struct. For instance, for the behavior we have JoinHandle<B> and when we await this, we get back B and place this into the Engine<B, E>.

I think this could probably be simplified, but it might not matter much since on top of this we can abstract away the user ever having to call these themselves by having a manager manage worlds (or something).